Issues in Conducting Empirical Evaluations of Controlled Languages

Simplified English (SE) is one of several restricted language standards that have been developed to reduce ambiguity and provide greater consistency and readability in technical documents. Proponents of SE (and other controlled languages) have claimed that using a restricted English standard makes documents easier to read and understand and easier to translate accurately into other natural languages. SE was designed to be applied to both procedural and descriptive writing, but in practice it has primarily been applied to procedural technical documents. The purpose of this paper is to discuss methods for assessing the claims about SE as used in procedural documents. First, we briefly overview the methodology and results of two experiments we conducted to test SE vs. non-SE documents. Next, we discuss some difficult and important issues that arose in designing and conducting the experiments and analyzing the results. Finally, we conclude with some recommendations for further empirical studies of SE.